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>> One of the perennial challenges in education and human development is how to make the most of 
information — whether test results, research findings, or other data. How do we ensure that information 
is accurate, relevant, and timely? What happens with all the data generated bytesting students? How do 
the results translate into improvements in practice? 



These challenges go to the heart of WestEd’s mission of promoting equity, excellence, and im- 
proved learning for children, youth, and adults. Accomplishing this mission means finding ways 



Alert — generating and using data that drive improvements. 

With this issue, we are proud to introduce a fresh new design for R&D Alert, shifting to more of 
the look and feel of a magazine, including full color photos, a new logo, and more article content. 



sessments that provide valuable data. One of these articles describes a new tool from WestEd’s 
Program for Infant/Toddler Caregivers (PITC) to assess the quality of early care settings and to 
identify areas where those settings may need improvement. 

Another article describes the development of the first new framework in 15 years for the science 
portion of the National Assessment of Educational Progress (NAEP). This framework will guide 
test developers in creating assessments that will be given nationwide in 2009 and beyond. As 
such, it is “the vehicle that drives much of our nation’s conversation about science education.” 

WestEd’s Stanley Rabinowitz discusses the challenge of ensuring that assessments and account- 
ability systems are of high enough quality for states and districts to depend on them in meeting 
the ambitious goals of No Child Left Behind. He describes the new Assessment and Accountability 
Comprehensive Center that he directs and that aims to address this challenge. 

Two other articles focus on using data effectively to inform improvement efforts. One describes 
the development of a new WestEd resource that helps build schools’ and districts’ capacity for 
data-driven decision-making. Another article shares lessons from WestEd’s work with districts on 
data-driven improvements that specifically address the education of English language learners. 

With well-designed assessments and wise use of data, educators, early care providers, and 
policymakers can make significant, lasting improvements that have enormous benefits for children. 
We hope the information and ideas in this newsletter are helpful in this regard, and we encourage 
you to pursue these topics further by using the contact information at the end of each article. 



to make information and analysis as useful as possible, and that is the focus of this issue of R&D 



Three of the articles focus on the generating side of our theme for this issue: the creation of as- 
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A COMPREHENSIVE PROCESS FOR SCHOOL IMPROVEMENT 
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Using data for 
decision-making 

» Even when school leaders have the will to improve and have all the data necessary to see where 
improvement is needed, they may still founder. What they often lack, says WestEd Senior Program 
Associate Lori Van Houten, are specific tools and an overall process for using data effectively. 



Van Houten and WestEd colleagues devel- 
oped such a process and compiled it in a 
new publication, Developing an Effective 
School Plan: An Activity- Based Guide to 
Understanding Your School and Improving 
Student Outcomes. 

Educators in California’s Beaumont Unified 
School District began piloting the process 
in 2004 and have already reaped impres- 
sive results. Chavez Elementary School, 
where the district focused much of its ef- 
fort, jumped 62 points on California’s Aca- 
demic Performance Index (API). Moreover, 
the low-socioeconomic subgroup targeted 
by the school’s data team had the greatest 
growth of all student subgroups, helping 
make Chavez the biggest success story 
in a district now celebrated for its overall 
academic turnaround. 



“When we put the tools in teachers’ hands to interpret 
and make sense of their own data, [those data] became 
much more meaningful to them,” says Darrell Brown, 
Beaumont’s Assessment and Accountability Coordinator. 
“And teachers took ownership of the process.” 

Developing an Effective School Plan provides a set of orga- 
nized activities and resources to help educators identify, 
collect, interpret, and use student achievement data to fuel 
school improvement. The process initially informs planning at 
the school and district levels, then moves to implementation. 

“Now they have a process and tools for getting ready to 
use data, analyzing student achievement data, discover- 
ing root causes behind those data, selecting effective 
practices, developing a plan, and then actually imple- 
menting and monitoring the plan,” says Van Houten, who 
worked closely with Beaumont. “A lot of schools never get 
that far. They spend all this time developing a plan, and it 
just keeps the binders from falling off the shelf.” 
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The big picture 

WestEd’s work with low-performing schools and districts 
has reaped significant results in schools from Massachu- 
setts to Hawaii. Through California’s Immediate Inter- 
vention/Underperforming Schools Program (II/USP), for 
example, WestEd has supported more than 75 schools in 
planning and implementing improvement plans. Over 70 
of those have sustained growth sufficient to exit the pro- 
gram and avoid state sanctions — a higher success rate 
than that of II/USP schools statewide. 

Such success informed the resources in Developing an 
Effective School Plan . During her work with Beaumont, 
however, Van Houten realized teachers need additional 
guidance to apply the planning and implementation 
activities to their own situation. 



Along with the Change Manager’s Hand- 
book, Developing an Effective School Plan 
contains facilitation notes for all the 
activities as well as a CD that includes 
all the activities and tools. The complete 
package is coauthored by Van Houten 
with WestEd Program Associate Kim 
AguIIard and with Jeanne Miyasaka, a 
consultant formerly with WestEd. The 
Western Regional Educational Laborato- 
ry at WestEd funded most of the devel- 
opment of the final product. 



When we put the tools in 
[those data] became much 




“They kept asking for the big picture,” she recalls. “There 
was one teacher who kept saying, ‘Where are we sup- 
posed to be going? What’s the big picture?’ And that’s 
when we figured out we had to have a way to look at the 
overall process so you can mesh these activities and tools 
with what already exists in your district, really build a 
road map for improvement.” 

The result is a “Change Manager’s Handbook,” which 
became the first volume in Developing an Effective 
School Plan . In addition to helping schools and districts 
take stock of their change process, the Handbook helps 
change managers identify other individuals who can help 
lead the process — people who have data content knowl- 
edge and the facilitation experience to lead the activities 
to a successful conclusion. 



Based on a wide body of research as well as 
lessons learned in the field, the publication 
lays out an ongoing process by which edu- 
cators use data as the basis for decision- 
making and achieving results at every level 
of the education system. [See Figure 1.] 

Comprehensive and ongoing 

Embedding activities and tools into a com- 
prehensive planning and implementation 
process was key to success in Beaumont. 

In cooperation with WestEd, the district 
conducted “Data Mentoring Workshops” 
for elementary sites. Teachers learned 
how to disaggregate results, spot trends, 
and adjust instruction accordingly — all 
while contributing to the creation and 
implementation of a comprehensive 
school plan for improvement. One hundred 
percent of teacher participants reported 
that they would be able to “effectively 
contribute to school improvement” as a 
result. School culture changed quickly. 
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“Teachers were actually anxious to see their data,” Van Houten says. 
“They were lined up outside the principal’s office, asking, ‘Are the data 
in yet? Can we see how we did?’ No one there had been interested in 
this before.” But changes by the teachers turned around the school’s 
previously disappointing performance, raising achievement most 
significantly among minority and low socioeconomic groups, she says. 
Similar changes occurred throughout the district. 

In one year, Beaumont went from declining API scores to the greatest 
growth of all districts countywide and the top 11 percent statewide. 
Five of the six schools targeted experienced significant growth in API 
scores, with a mean of 47 points. The district also experienced an 8.3 




To order Developing an Effective School Plan, visit 

www.WestEd.org/schoolplan. 



teachers’ hands to interpret and make sense of their own data, 

more meaningful to them . . . . And teachers took ownership of the process. 



Figure 1. The Inquiry Cycle 




4 > Determine Effective Practices 
and Write a Plan 



• What are our priorities? 

• What specific practices will we use 
to address our identified needs? 

• What is our plan for school 
improvement? 



2 > Organize and Analyze Data 



• What is our student achievement? 

• What patterns and trends are in 
our achievement data? 

• On what areas will we focus our 
school improvement efforts? 



3 > Investigate Factors Impacting 
Student Achievement 



How do our curriculum, 
instruction, and assessments 
affect student achievement? 
What can we learn from student 
and teacher demographics? 
What can we learn from 
perception data? 



percentage-point improvement in Eng- 
lish language arts and a 10.1 percentage- 
point gain in math. 

Brown recommends the data-driven ap- 
proach to any school system. “You don’t 
wait till you’re in trouble to improve — you 
want to be doing improvement all the time,’ 
says Brown. “We’re eager to continue.” 






For more information, contact 
Lori Van Houten at 415.615.3165 
or Lvanhou@WestEd.org. 



07 






English learner evaluation 
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accountability support 



■The 



Su **sh 



*ne 



SOme of the largest, 




Iemo; 



oranges, and avocados 



,n the state. 



Today, with WestEd’s help, Coachella Valley Unified School District is tending 



to 



another 



abundant resource 



: its 



English language 



learners. 



A third of the nation’s almost five million English language learners are 
in California schools. There is immense pressure to help these students 
gain English proficiency. Moreover, Proposition 227, passed by voters in 
1998, requires that English learners in California public schools be taught 
“overwhelmingly in English.” 

In Coachella Valley, a vibrant Latino community has grown with the area’s 
agricultural and service jobs. The school district is 97 percent Latino, with 
a steady churn of new arrivals who bring various levels of literacy and 
English speaking ability. 

“Every teacher in our district is an English Language Development teach- 
er,” says Alma Gonzalez, English learner services director for Coachella 
Valley Unified. “Our proportion of English learners is about 70 percent.” 



Systematic use of data to drive improvement 

WestEd’s Robert Linquanti and Linda Carstens caught the attention 
of Coachella Valley’s superintendent during an institute for local 
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continued from education leaders. Through WestEd’s 
previous page English Learner Evaluation and Account- 
ability Support (ELEAS), Linquanti and 
Carstens help districts improve the suc- 
cess of students for whom English is not 
the primary language. 

ELEAS is informed in part by what Lin- 
quanti, collaborating with the American 
Institutes for Research, learned from an 
extensive review of the impact of Califor- 
nia’s Proposition 227. The review found, 
among other things, that “systematic, 
ongoing assessment and careful use of 
data to guide instruction” were among the 
elements most critical to helping English 
learners succeed. 



“Improving English language development 
and understanding for English learners 
is a systemic challenge,” adds Linquanti, 
who directs ELEAS. “It requires districts, 
schools, and teachers to establish some 
key goals; take a pragmatic, data-wise ap- 
proach to achieving them; and cultivate in- 
ternal accountability to get the job done.” 




And because it asks districts to change 
their culture, “it’s not easy.” He emphasiz- 
es that educators need to start by know- 
ing where they are. 

In Coachella Valley, state test 
scores reflected some of 
the district’s challenge. 
Less than 14 percent 
of the district’s el- 
ementary and middle 
school students met 
or exceeded state 
English and language 
arts standards in 
2003 — 04, and only 
about 16 percent met 
math standards. Scores 
at the high school level were 
similarly low. 



Coachella Valley educators knew they needed outside as- 
sistance to improve English language development, recalls 
Gonzalez. The district received a two-year, Title I Supple- 
mental grant and asked WestEd to help set the course. 

At the heart of WestEd’s ELEAS approach is using data 
analysis to identify critical instructional issues; estab- 
lishing a few concise, districtwide goals; and engaging in 
timely disaggregation of data to guide teachers in adjust- 
ing materials or methods to meet the needs of students. 

WestEd and Coachella educators looked closely at test 
scores and demographics, paying attention to how English 
learners fared over time in the district. “We found that 
some of our students just were not making the right prog- 
ress and seemed stagnant,” Gonzalez says. “Their English 
language development wasn’t where it needed to be. In 
some cases, though, their academic achievement was at 
par or higher than English-only speakers.” 

After refining their understanding of the data, district 
leaders worked to develop goals to help all English learners 
in the schools see tangible improvements in both English 
language development and core academic classes. 

From philosophy to practice 

ELEAS’s Carstens says translating data analysis into long- 
term systemic changes requires districtwide participation 
in developing “the bigger picture.” 

“You start with philosophy: What do you think about 
English learners, who they are, what they can learn? 

Then, what kind of teaching supports that philosophy?” 
Carstens says. “Only then can you get down to pedagogy 
and the practices that support that pedagogy.” 

Conversations among Coachella’s leaders and educators 
led to a shared understanding of district philosophy and 
an overall goal of having every English learner make one 
level of growth per year in both English language develop- 
ment and in academic areas. 

The district also worked to disaggregate data in order to 
understand where each student was starting from, how 
well students do in relation to how long they are in the dis- 
trict, and what instructional areas need strengthening. 

Disaggregating data “is not a problem-solving strategy. 

It’s a problem- finding strategy,” notes Linquanti. It can 
help determine professional development priorities to 
improve teaching practices. 
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The study on the "Effects 
of the Implementation of 
Proposition 227 on the Edu- 
cation of English Learners, 
K-12” is available from 
www.wested.org/prop227. 



Coachella’s process resulted in adjustments to the dis- 
trict’s teacher-guided approach to learning. Gonzalez 
says, “We had to make sure students were given time to 
interact with the language.” The district’s emerging phi- 
losophy has been translated into more specific practices 
that encourage students to share ideas, read aloud, and 
engage in discussions. 

All of these practices help students become more versatile 
with their new language, but implementing these prac- 
tices is a change for many teachers, Gonzalez says. “Their 
classes are not going to be so quiet.” 

Continuous improvement 

As Coachella educators are getting up to speed with 
ELEAS, just hours away in Orange County’s Garden 
Grove Unified School District, teachers and administra- 
tors who have been working with ELEAS since the spring 
of 2002 are seeing their efforts pay off. 

“The goals have become integrated into everything we 
do,” says Debbie Youngblood, director of K-12 educational 
services for the district. District leaders established two 
goals for this large, urban district with significant num- 
bers of Asian and Hispanic English learners: to make mea- 
surable progress in academic proficiency and to advance 
English language development and proficiency. 

Youngblood says teachers have been included from early 
in the process. Regular training in how to use new data 



Disaggregating data is 
not a problem-solving 
strategy. It’s a problem- 
finding strategy 





tracking tools has helped teachers become comfortable 
using data analysis as a routine part of instructional plan- 
ning. Quarterly benchmark tests are used, in addition to 
statewide standardized tests. 

After three years of working with Linquanti’s “goal-driv- 
en, data-wise, research-based” approach, teachers in the 
Garden Grove district have become adept at looking to 
assessment results. They are asking for more data, more 
quickly, to guide their work with English learners. 

For administrators, this means crafting data reports 
that respond to teachers’ needs — quickly. The data have 
helped the district choose new programs to offer. And 
data help on the individual level by revealing what sub- 
skills should be emphasized to help a student who has 
plateaued or lost ground. 

In 2004, Garden Grove won the $500,000 Broad Prize 
for Urban Education, which recognizes districts for 
improving student achievement and reducing achieve- 
ment gaps among ethnic groups and between high- and 
low-income students. 

Garden Grove has made the transition into a district that 
naturally incorporates data analysis into its planning 
process, but the challenges never really end. Says Young- 
blood, “It’s a constant discussion of how we can improve 
what we’re doing.” 

For the students, those improvements may build the bridge 
between learning English, and learning in English. 



:o 



For more information, contact Robert Linquanti 
at 510.302.4235 or rlinqua@WestEd.org; or Linda 
Carstens at 626.836.7753 or lcarste@WestEd.org; 
or visit www.wested.org/cs/we/view/pj/367. 










No Child Left Behind lays out a funda- 
mental challenge, asking every school to 
ensure that every student achieves high 
standards. Teachers and administrators feel 
the pressure of this challenge every day. But 
another group that may feel the pressure of 
NCLB almost as strongly includes test de- 
velopers and the policymakers who oversee 
each states accountability program. 




Creating Accountability Systen 



Assessments and accountability systems play a pivotal 
role in the implementation of NCLB. They must be of 
high quality for the ambitious goals of NCLB to be met. 

To help states address this challenge, the U.S. Depart- 
ment of Education in 2005 selected WestEd, in partner- 
ship with the National Center for Research on Evalu- 
ation, Standards, and Student Testing (CRESST), to 
operate a new kind of assistance center — the Assess- 
ment and Accountability Comprehensive Center (AACC). 

WestEd s Stanley Rabinowitz directs the new center. 

He has conducted research, published papers, and con- 
sulted with state and local districts on issues related 



What is the motivation behind creating an Assessment 
and Accountability Comprehensive Center? 

Assessment and accountability systems are key to the suc- 
cess of No Child Left Behind. They provide the information 
that everyone — states, districts, schools, teachers, stu- 
dents — must use to guide improvement. They signal goals, 
focus instruction, and are fundamental tools for achieving 
the overarching aim of academic proficiency for all students. 

The AACC’s role is to help states make sure that their assess- 
ment and accountability systems are up to the task and that 
local schools have the tools they need to improve instruction 
and other education programs. 



to NCLB accountability and assessment provisions, 
including the use of integrated standards and assess- 
ment systems in high-stakes state accountability pro- 
grams; high school exit exam policies for students with 
disabilities; and assessment of English language learn- 
ers on core academic content. Prior to joining WestEd 
in 1991, Rabinowitz directed the statewide assessment 
program for the New Jersey Department of Education. 

R&D Alert recently spoke with Stanley Rabinowitz 
about his work and the new Assessment and Account- 
ability Comprehensive Center. 



How is the AACC different from other 
assessment-focused groups? 

The U.S. Department of Education sees us as field agents 
working on the most important and difficult assessment and 
accountability issues under NCLB. That’s our challenge. 

We work with the Regional Comprehensive Centers, and in 
some instances, state departments of education — not di- 
rectly with districts and schools. Our work is designed to ben- 
efit local districts and schools by making sure the states are 
putting into practice the best knowledge on how to improve 
and support their assessment and accountability systems. 
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and accountability systems, not developing new products 
or programs. We’re using our expertise to review and evalu- 
ate what’s already out there and make resources available 
in user-friendly formats, mainly through websites and other 
dissemination strategies. 

Could you give an example of a resource you are or 
will be using? 

One is a set of technical guidelines, titled The Technical Ad- 
equacy of Assessments for Alternate Student Populations: 
Guidelines for Consumers and Developers, which I developed 
with WestEd’s Edynn Sato and an expert panel under a previ- 
ous contract from the U.S. Department of Education. 

What is significant about these guidelines? 



In a broader sense, we’re also trying to inform the improvement 
of assessment development and accountability systems as 
they move forward. 

What are some of the "most important and difficult” is- 
sues identified so far? 

For our first year, the AACC has three main areas of focus. 

One is assessment and accountability relating to special 
populations — how to help narrow the achievement gap, par- 
ticularly for special education students and English learners. 

The second piece is what we’re calling our data use strand, 
which focuses mainly on the classroom and local level. How 
can teachers get the information they need from state and 
diagnostic assessments to improve instruction? How can ad- 
ministrators get the information they need to make program 
and accountability decisions? 

Andthethird piece isa systems focus — working with states to 
improve their overall assessment and accountability programs. 
We will expand to other areas of focus in subsequent years. 

How will you go about this work? 

Mostly through a combination of finding, reviewing, and 
synthesizing the available research products and services 
that states and schools can use to improve their assessment 



The number of English learners is increasing, and NCLB has 
raised the assessment and accountability expectations and 
requirements for this population. So, this is really a grow- 
ing area of concern, and states are scrambling to develop or 
acquire assessments for English learners. 

We felt there was a real need for an external, expert, neu- 
tral party to review the evidence about these assessments 
— their validity, reliability, and freedom from bias. More 
importantly, we wanted to be able to inform states and other 
consumers about what to look for when evaluating such 
tests, so they can make informed decisions about which ones 
to choose, and have more confidence in the results once the 
assessments are selected and administered. 

It sounds like the guidelines are about assessments for 
English learners, not necessarily other populations too? 

We do talk about how the procedures can generalize, but 
they need to be applied somewhat differently for other 
populations. And one thing we’ll do over the five-year 
funding period for the AACC is continually update and 
expand the guidelines, which will mean looking at as- 
sessments for students with disabilities and other groups 
besides English learners. 






» Mapping the human genome, monitoring ozone layer depletion, 
tracking mutations of viruses capable of spreading pandemic flu, 
building satellites that can circle the planet in an hour: These and 
other scientific breakthroughs have Ranged the world at the 
start of the 21st century. And Americans, including our students, 
must be able to keep up with the accelerated pace of knowledge. 
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To measure progress on this front, WestEd 
recently spearheaded the development of a 
new science education assessment frame- 
work that reflects the most current think- 
ing in the fields of physical, life, and earth 
and space sciences. 



Through a contract with the National As- 
sessment Governing Board, WestEd and 
the Council of Chief State School Officers 
brought together leading researchers, 
educators, and policymakers to provide 
recommendations for the first update in 
15 years of the National Assessment of 
Educational Progress (NAEP) framework 



The si nificance of the NAEP framework 
cannot be overstated - . . . This is the vehicle 
that drives much of our nations conversation 
about science education - 




for science assessment. The new frame- 
work will help test developers create na- 
tional assessments for science in 2009 and 
beyond and could have a major impact on 
the standards and assessments that states 
adopt to comply with requirements of the 
federal No Child Left Behind Act. 



Second, it explains the thinking and rea- 
soning processes (cognitive demands) that 
students engage in to complete various 
assessment tasks. And third, it expands 
the range of items that test developers 
can use to probe students’ understanding 
of science. 



“The significance of the NAEP frame- 
work cannot be overstated,” says Gerald 
Wheeler, executive director of the National 
Science Teachers Association and chair 
of the NAEP science framework steering 
committee. “This is the vehicle that drives 
much of our nation’s conversation about 
science education.” 

The framework “breaks new ground” in sev- 
eral ways, says Richard J. Shavelson, Marga- 
ret Jack Professor of Education at Stanford 
University and co-chair of the planning com- 
mittee that coordinated the project. First, the 
framework links science concepts to scien- 
tific practices in order to create performance 
expectations (see Figure 1, page 16). 



For example, computer-assisted tasks could 
simulate the life cycle of plants and then 
ask students to reflect on the observable 
relationship between sunlight and germina- 
tion. Interactive CDs could replicate labora- 
tory experiences that may be too dangerous 
or messy for fourth-graders to conduct in 
the classroom but could help them dem- 
onstrate what they know about recording 
data and drawing conclusions. Middle and 
high school students could use a customized 
software program to create concept maps 
that show their understanding of the rela- 
tionships among important science ideas 
that lead to solving problems. 










Figure 1. Crossing Content and Practices to Generate Performance Expectations 
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“We focused on the uses of science 
principles,” says Senta Raizen, director 
of WestEd’s National Center for Improv- 
ing Science Education and co-chair of the 
NAEP science framework planning com- 
mittee. “It’s about predicting, observing, 
and explaining science phenomena — 
using science knowledge rather than just 
identifying terms.” 

The NAEP science framework builds on 
the National Science Education Standards, 
published by the National Research Coun- 
cil in 1996, and the Benchmarks for Scien- 
tific Literacy, published by the American 
Association for the Advancement of Sci- 
ence in 1993. In addition, the framework 
reflects international assessment stan- 
dards, such as those represented in the 
Trends in International Mathematics and 
Science Study (TIMSS). 



But in setting the course for future U.S. 
science assessments, the NAEP designers 
carefully reduced the number of standards 
and content statements to encourage test 
developers to focus on depth, not just 
breadth, of knowledge. 

An overview chapter adds, “the Framework 
attempts to strike a balance between what 
can reasonably be predicted about future 
school science and what students are likely 
to encounter in their curriculum and in- 
struction now and in the near future.” 



The framework focuses on predicting , 
observing, and explaining science 
phenomena — using science knowledge 
rather than just identifying terms - 




A prepublication version of 

the framework is available free online: 

www. nagb.org/pubs/ 
sci_fw_prepub_0 1 1 9. pdf 



The framework recommends specific 
science concepts in the physical, life, and 
earth and space sciences to be assessed at 
grades 4, 8, and 12, and four major catego- 
ries of science practices: 

§ identifying science principles (e.g., 
students may be asked to choose the 
correct response to a question about 
what happens to plants and animals 
when they die) 
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§ using science principles (e.g., students 
may be asked to draw on their knowledge 
of the earths tilt and its rotation around 
the sun to explain the change of seasons) 

§ using scientific inquiry (e.g., students 
may be asked to design an experiment 
that would assess factors affecting 
plant growth, assess whether the re- 
sults of the investigation are consistent 
with theoretical models, and critique 
their investigation) 

§ using technological design (e.g., stu- 
dents may be asked to recommend a 
new energy plan for a community and 
evaluate the environmental impacts of 
natural gas versus coal) 

“It’s critical to embed the science practices 
in the science content,” says Alice Fu, a 
WestEd research associate who helped 
write the NAEP framework. “Anything you 
expect from students should draw on both 
content and practice.” 

Working under tight time constraints and 
balancing diverse views about science 
education, the NAEP framework design- 
ers achieved a remarkable level of con- 
sensus about what students should know 
about various topics. The project’s success 
stemmed in large part from the ongoing 
involvement of major stakeholders at fed- 
eral, state, and local levels. 



“It was clear early on that WestEd brought to the process 
a class act’ in the way it staffed the project and carried it 
out in a seamless manner,” Shavelson says. 



In addition to the regular committee and subcommittee 
work, organizers conducted a national meeting for repre- 
sentatives of the Council of State Science Supervisors, 13 
regional meetings for representatives of various scientific 
communities across the country, an Internet survey of 
science teachers in conjunction with the National Science 
Teachers Association, and presentations at national and 
regional professional conferences. At each step in the 18- 
month development process, organizers sought reactions 
and addressed concerns. 

“From beginning to end, we’ve been proactive about getting 
feedback from many sources. That helped act as a check 
on our work,” Fu says. “We ended up with a product that 
represents the perspectives of a lot of different groups and 
is also current and forward-looking. It’s sort of amazing.” " 




For more information abut the NAEP science frame- 
work, contact Senta Raizen at sraizen@WestEd.org 
or Alice Fu at afu@WestEd.org. 
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Assessing Early 
Childhood Care 



» Measuring how well early 
childhood programs use 
concepts and practices that 
promote quality care for 
infants and toddlers has just 
gotten easier, thanks to a new 
tool developed by WestEd’s 
Program for Infant/Toddler 
Caregivers (P1TC). 



The tool, known as the PITC Program Assessment Rating Scale (PITC 
PARS), represents the latest development in PITC’s 20-year mission to 
train infant care providers to deliver care that reflects the latest research 
findings in the area of early development and care. Closely aligned with 
PITC concepts, practices, and policies, PARS is designed to assess quality 
of care before and after PITC training. 

“We see PARS being used in tandem with other widely used measures of 
quality care,” says Peter Mangione, co-director of WestEd’s Center for 
Child and Family Studies, which oversees PITC. “Other measures give you 
an overall sense of how much progress you’re making, according to widely 
shared and accepted standards. PARS, on the other hand, lets you docu- 
ment more specific dimensions of infant and toddler care that the Pro- 
gram for Infant/Toddler Caregivers identifies as being very important.” 

One of the most important, according to the research, is child care that 
emphasizes the development of strong, positive relationships between 
infants or toddlers and their earliest caregivers. Such relationships pro- 
mote feelings of safety and security that, in turn, allow young children to 
attend to intellectual pursuits. 



Kerry Kriener-Althen, a WestEd Senior Research Associate, points out 
that assessing the quality of relationship-based child care is difficult. Yet, 
such assessment is critical. “You can have all the best equipment, toys, 
and books in the world, but to provide optimal care, teachers need to be 
truly engaged with the infants and toddlers and effectively supporting 
their interests and learning.” 
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continued from Mangione agrees. PARS is a unique as- 
previous page sessment tool, he says, because it places 
a “greater emphasis on the relationship 
between caregivers and children than one 
typically sees in instruments designed to 
measure infant and toddler care” 

PARS is available in two versions: one for 
use in family child care settings, the other 
for center-based care. Both versions are 
structured to assess five main components 
of infant-toddler care, as defined by PITC: 

§ The quality of caregivers’ interaction 
with infants 

§ Family partnerships, cultural 
responsiveness, and inclusion of 
children with disabilities 
§ Relationship-based care 
§ Physical environment 
§ Routines 



Evetything from the 
way a child is helped to 
fall asleep at naptime 
to ways of holding and 
carrying a child are part 
of a family’s particular 
culture Providing 
responsive care means 

being responsive to that 
culture. 



Each component, in turn, is divided into 
several items. For example, the quality 
of caregivers’ interactions with infants is 
assessed by measuring such things as the 
caregivers’ responsiveness and sensitivity 
to children, respect for infants’ initiatives 
and choices, and facilitation of language 





development and communication. The quality of 
family partnerships is measured by the extent to 
which the program promotes positive communication 
with families as well as care that is culturally consistent 
with and supportive of children’s home environments. 
Relationship-based care is assessed by considering the 
quality of primary caregiving, the continuity of care, and a 
caregiver’s ability to follow a child’s individual schedule. 

Finally, each item consists of four sub-items, each of 
which is scored as either “met” or “not met,” depending 
on information gained through observation and interview. 
To measure a caregiver’s responsiveness and sensitivity 
to children, for instance, an evaluator must determine 
whether or not a caregiver responds promptly to chil- 
dren’s distress cues. To do so, the evaluator must watch 
for instances in which children exhibit signs of distress 
(such as crying, fearful expressions, or agitation) and 
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then rate the consistency and timeliness of 
the caregivers responses. A User’s Guide has also 
been developed to promote consistent scoring of sub- 
items across evaluators. 



Kriener-Althen says that measuring whether or not care 
is “culturally consistent” could involve something as 
simple as documenting a teacher respecting a parent’s 
wishes about food at snack time. “Everything from the 
way a child is helped to fall asleep at naptime to ways of 
holding and carrying a child are part of a family’s particu- 
lar culture,” she says. “Providing responsive care means 
being responsive to that culture.” 



Kriener- 
Althen asked. 

Of course, infant care 
teachers can’t be expected to 
respond to every child immediately. 
“But if a teacher doesn’t respond, is it because 
that teacher is working with another child, or talk- 
ing to another teacher?” asked Kriener-Althen. “A teacher 
could be sitting across the room from a child who is voicing 
a concern and begin to address that concern simply by say- 
ing, ‘I’ll be right there.’” 



Given the fact that PITC, the signature project of 
WestEd’s Center for Child and Family Studies, is the single 
largest infant-toddler training program in use nationwide, 
Mangione expects PARS to be widely used. “Thousands 
of child care providers around the country are running 
programs that implement PITC concepts and practices,” 
he says. “We see PARS as a tool to give them a way to look 
more comprehensively and specifically at the care they’re 
giving.” What’s more, Mangione says PARS will also help 
PITC trainers assess whether their training is achieving 
what they want. “We want to know, are we getting the 
kind of responsive, relationship-based care from caregiv- 
ers that we’re training for, and can we document it? We 
think PARS will help us answer that question.” 



Both Kriener-Althen and Mangione say that PARS evalu- 
ators were asked to “put themselves in the place of the 
children” they were observing. “Would you feel your con- 
cerns were being addressed promptly and appropriately?” 




For more information, contact Peter Mangione at 
415.289.2310 or pmangio@WestEd.org, or Kerry 
Kriener-Althen at 415.289.2338 or 
kkriene@WestEd.org. 
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Developing an Effective 
School Plan: An Activity- 
Based Guide to Understand- 
ing your School and Improv- 
ing Student Outcomes 

The inquiry, planning, and imple- 
mentation process embodied in 
this school improvement pack- 
age represents the distillation 
and thoughtful organization of 
what WestEd staff have learned 
firsthand from their work with 
diverse schools across the country. 
Developing an Effective School 
Plan includes a facilitation guide, 
activities, interactive tools, and 
CD-ROM. The tools and activities 
represent the most effective on- 
site processes for guiding schools 5 
improvement efforts. 

Resiliency: What We 
Have Learned 

This synthesis of more than a 
decade of resiliency research 
highlights the role that families, 
schools, and communities can 
play in supporting children's 
and youth's natural capacity to 
lead healthy, successful lives. Of 
special interest is the evidence 
that resiliency prevails in most 
cases by far — even in extreme 
situations, such as those caused 
by poverty, troubled families, and 
violent neighborhoods. Benard 
offers a practical analysis of 
how best to incorporate research 
findings to support young people. 



Nine Lessons of Successful 
School Leadership Teams: 
Distilling a Decade of 
Innovation 

Why do some school leader- 
ship teams succeed while others 
stagnate, snipe, or disintegrate? 
Are there key lessons that apply 
no matter what your school situ- 
ation? Nine Lessons of Successful 
School Leadership Teams distills 
a decade of on-the-ground in- 
novation and research pointing 
to what school leadership teams 
can do to focus on and increase 
student achievement. 

Central Office Inquiry: 
Assessing Organization, 
Roles, and Functions 
to Support School 
Improvement 

Schools workingto raise student 
achievement need the help of the 
central office. Yet many districts 
lack unified direction, agree- 
ment on the central office role, 
and coherence and alignment 
between goals and strategies. 

This book is intended to help cen- 
tral office leadership and staff 
examine their organizational 
arrangement, enacted roles, and 
day-to-day activities, critically 
questioning both their theories 
of action and how their work is 
helpingthe schools they serve. 
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Making Science Accessible 
to English Learners: 

A Guidebook for Teachers 

This guidebook is for science 
teachers, particularly in upper 
elementary, middle school, and 
high school, who have ample 
knowledge of science standards 
and concepts, are comfortable 
with basic teaching and class- 
room management, but may 
have had limited preparation for 
teaching science in classrooms 
where at least some students are 
also English learners. The book 
offers practical guidance and 
powerful, concrete ways to help 
English learners. 



Inside High School 
Reform: Making the 
Changes That Matter 

What happens when some of the 
lowest-performing high schools 
in California make a commitment 
to reform themselves? This book 
goes inside the reform efforts 
of 28 schools where educators 
collaborated to fundamentally 
change expectations for students 
— in effect, to prepare all stu- 
dents for postsecondary educa- 
tion. In the words of the educa- 
tors themselves and through the 
perspectives of advisors who 
monitored the reform programs, 
this book lays out some of the 
apparently universal lessons of 
making the changes that matter. 
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Can you give a sense of what these 
guidelines say? 

There are certain categories of evidence 
that should be considered when evaluat- 
ing any test — mostly concerning reliabil- 
ity, validity, and freedom from bias. So, 
the guidelines we developed talk about 
tailoring how you look at these categories 
for assessments of special populations. 

You have to look at reliability, valid- 
ity, and bias in different ways for English 
learners. Take validity, for example. For 
tests in general, you might check con- 
tent validity by having content experts 
review the test — mathematics experts 
for the math items, and so on. But for a 
test that will be used with English learn- 
ers, you need to have reviews by those 
content experts — the math people and 
others — plus you need to have experts in 
linguistics and educators of English learn- 
ers review the test. You need to make sure 
that the content and the expectations for 
the given student population are right. 



We felt there was a real need for an 
external, expert, neutral party to 

review the evidence about these 
assessments — their validity, 
reliability, and freedom from bias. 



Who will use these guidelines? 

Primarily test developers and consumers — the consumer 
here isn’t a parent, it is a state or a district. So, if I am a 
consumer, the guidelines provide categories and ways of 
evaluating a test before I purchase it or for improving it 
once I have it. If I’m a developer, the guidelines help me 
consider issues I might have overlooked in analyzing my 
test to make sure it is valid, reliable, and free of bias. 

Another audience is broader, though. We want to be able 
to inform assessment development as the field moves 
forward. We are trying to guide the next generation of 
tests that will be used to gauge whether students are 
reaching the standards set by states. 

In reviewing the current generation of high-stakes tests, 
our sense is that they need to provide better access for 
English learners, students with disabilities, and other 
special populations. We expect the tests to get better as 
time goes on. But there’s a lot of work to be done to get 
there, and we hope the AACC will help in that process. 






For more information, visit www.aacompcenter.org or 
contact Stanley Rabinowitz at 415.615.3154 or 
srabino@WestEd.org. 
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